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(57) For an apparatus effectively providing a real 
three-dimensional model to a customer, three-dimen- 
sional shape data of a target is obtained at a data input 
section (1 0). The data is managed by a server which is 
connected to a network. A suitable formation section 
(14) is selected to form a real three-dimensional model 
based on the data. In addition, the formation completion 
is automatically notified to the customer via a completion 
display panel (260), a cellular phone (61), or the like. 
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Description 

Technical Field 

[0001] The present invention relates to a three-dimen- 
sional modeling apparatus for supplying a three-dimen- 
sional model of the target based on three-dimensional 
shape data of a target. 

Background Art 

[0002] Various types of three-dimensional scanners 
have been known for years, which can be used to obtain 
three-dimensional shape data of various types of ob- 
jects. Engraving machines which are driven based on 
the three-dimensional shape data are also available. A 
real three-dimensional model can be produced using 
such an engraving machine. 

[0003] A system is known in which three-dimensional 
shape data of a target such as a person or a statue is 
obtained by irradiating laser ray on the target and pro- 
ducing the target using the engraving machine. 
[0004] However no apparatus for producing a real 
three-dimensional model and providing the model to a 
customer is commercially available yet. One of the prob- 
lems of such an apparatus is that a long period of time 
is required from the input of the three-dimensional 
shape data to the production of the real three-dimen- 
sional model to such an extent that it was impossible to 
prepare the model without making the customer feel un- 
pleasant. Moreover, there has been no proposal of any 
preferable means for notifying the customer of the com- 
pletion of the model production. Furthermore, there has 
been no method for managing the considerable 
amounts of three-dimensional shape data and for effec- 
tively and efficiently showing the data to the customer. 

Disclosure of Invention 

[0005] The present invention is conceived to solve the 
above described problem and one object of the present 
invention is to provide a practical system for effectively 
and efficiently providing a real three-dimensional model 
and three dimensional shape data to a customer. 
[0006] In order to achieve the object described above, 
according to one aspect of the present invention, there 
is provided a three-dimensional modeling apparatus for 
supplying a three-dimensional model of a target to a 
customer. This apparatus comprises an input section for 
inputting three-dimensional shape data of the target; a 
formation section for forming a real three-dimensional 
model based on the obtained three-dimensional shape 
data; and a recording medium issuing section for issuing 
a recording medium in which information can be written 
to or read from, when the three-dimensional shape data 
is input, wherein the recording medium can be read by 
the three-dimensional modeling apparatus and/or a 
third-party apparatus other than the three-dimensional 



modeling apparatus. 

[0007] According to an other. aspect of the present in- 
vention, it is preferable that the recording medium is ca- 
pable of recording data identification information. 
5 [0008] According to another aspect of the present in- 
vention, it is preferable that the recording medium allows 
new information to be written by the three-dimensional 
modeling apparatus and/or an third-party apparatus oth- 
er than the three-dimensional modeling apparatus. 
10 [0009] According to another aspect of the present .in- 
vention, it is preferable that the third-party apparatus is 
an entertainment device and that the new information is 
a score of the entertainment provided by the entertain- 
ment device, the score being evaluated by the entertain- 
15 ment device. 

[0010] According to another aspect of the present in- 
vention, it is preferable that the apparatus further com- 
prises a recording medium information re-input section 
for re-reading new information from the recording medi- 
co urn, the new information written by the third-party appa- 
ratus. 

[0011] According to another aspect of the present in- 
vention, it is preferable that the formation parameters 
are changed based on the information input from the re- 
25 cording medium information re-input section and a real 
object model is formed from the changed formation pa- 
rameters. 

[0012] According to another aspect of the present in- 
vention, it is preferable that three-dimensional shape 
30 data which is input by the input section is recorded on 
the recording medium. 

[0013] According to another aspect of the present in- 
vention, it is preferable that the third-party apparatus is 
an entertainment device which reads three-dimensional 
35 shape data recorded on the recording medium and ex- 
ecutes an entertainment program of the entertainment 
device. 

[0014] In order to achieve the objects described 
above, according to another aspect of the present in- 

40 vention, there is provided a three-dimensional modeling 
apparatus for supplying a three-dimensional model of a 
target to a customer, the apparatus comprising an input 
section for inputting three-dimensional shape data of the 
target; a formation section for forming a real three-di- 

45 mensional model based on the obtained three-dimen- 
sional shape data; a communication section which can 
communicate with the three-dimensional modeling ap- 
paratus and/or a third-party apparatus other than the 
three-dimensional modeling apparatus via a network; 

50 and a formation process progress information transmit- 
ting section for transmitting progress information of the 
formation process at the formation section to the appa- 
ratus. 

[0015] According to another aspect of the present in- 
55 vention, it is preferable that the formation process 
progress information of the formation section includes 
formation completing time information and the time in- 
formation is displayed at a display section provided on 
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the apparatus. 

[0016] According to another aspect of the present in- 
vention, it is preferable that the communication section 
receives the information at the apparatus and changes 
parameters for forming the real three-dimensional mod- 
el based on the information, and that the formation sec- 
tion forms the real three-dimensional model based on 
the changed parameters. 

[0017] According to another aspect of the present in- 
vention, it is preferable that the three-dimensional shape 
data is transmitted to the apparatus via the communica- 
tion section. 

[0018] According to another aspect of the present in- 
vention, it is preferable that the apparatus is an enter- 
tainment device and that the transmitted three-dimen- 
sional model shape data is used, by the entertainment 
device. 

[0019] In orderto achieve the object described above, 
according to another aspect of the present invenlion, 
there is provided a three-dimensional modeling appara- 
tus for supplying a three-dimensional model of a target 
to a customer, the apparatus comprising an input sec- 
tion for inputting three-dimensional shape data of the 
target; a formation section for forming a real three-di- 
mensional model based on the obtained three-dimen- 
sional shape data; a contacting means input section for 
inputting information contacting means for the custom- 
er; and an information notifying section for notifying 
modeling information to the information contacting 
means. 

[0020] According to another aspect of the present in- 
vention, it is preferable that the information contacting 
means for the customer is a cellular phone, the phone 
number for the cellular phone is input at the contacting 
means input section, and the modeling information is 
progress information of the formation process at the for- 
mation section. 

[0021] According to another aspect of the present in- 
vention, it is preferable that the information contacting 
means for the customer is electronic mail, the mail ad- 
dress for the electric mail is input at the contacting 
means input section, and the modeling information is 
formation completion information. 
[0022] According to another aspect of the present in- 
vention, it is preferable that the modeling information is 
identification information for accessing a server which 
stores the three-dimensional shape data, in addition to 
being the formation completion information. 
[0023] According to another aspect of the present in- 
vention, it is preferable that the modeling information is 
three-dimensional shape data. 

[0024] According to another aspect of the present in- 
vention, it is preferable that the apparatus further com- 
prises an estimated completion time display section for 
displaying the shortest estimated time for the formation 
completion and a formation completion time specifying 
section for specifying a time after the shortest formation 
completion time, wherein a schedule for the formation 



process is determined to accommodate the completion 
time indicated at the formation completion time specify- 
ing section by the customer. 

[0025] According to another aspect of the present in- 

5 vention, it is preferable that the apparatus further com- 
prises a process start receiving section for receiving 
start time for the formation process, wherein the forma- 
tion process is started according to the start time indi- 
cated by the customer. 

10 [0026] According to another aspect of the present in- 
vention, it is preferable that the apparatus further com- 
prises an address means Input section for inputting de- 
livery destination address forthe real three-dimensional 
model; and a delivery destination printing section for 

is printing the delivery destination address. 

[0027] According to another aspect of the present in- 
vention, it is preferable that the apparatus further com- 
prises a formation section selecting section, connected 
to a plurality of formation sections via a network, for se- 

20 lecting a suitable formation section based on the input 
address, wherein the three-dimensional shape data is 
transmitted to the selected formation section. 
[0028] According to another aspect of the present in- 
vention, it is preferable that the apparatus further com- 

25 prises a communication section which can communi- 
cate with a database section for storing the three-dimen- 
sional shape data via a network, wherein the three-di- 
mensional shape data is recorded at the database sec- 
tion. 

30 [0029] According to another aspect of the present in- 
vention, it is preferable that the apparatus further com- 
prises a three-dimensional shape data synthesizing 
section for reading three-dimensional shape data pre- 
stored in the database section and generating new 

35 three-dimensional shape data obtained by integrating a 
plurality of three-dimensional shape data. 
[0030] According to another aspect of the present in- 
vention, it is preferable that the apparatus further com-, 
prises a plurality of formation sections, and a server sec- 

40 tion for managing the three-dimensional shape data in- 
put from the input section , wherein the formation section 
is selected at the server section for assigning the three- 
dimensional shape data input from the input section. 
[0031 ] According to another aspect of the present in- 

45 vention, it is preferable that the selection of the forma- 
tion section is performed based on the operation condi- 
tions at the plurality of formation sections. 
[0032] According to another aspect of the present in- 
vention, it is preferable that the apparatus further com- 

50 prises a parameter specifying section for the customer 
to specify the material or size of the object to be formed , 
wherein the selection of the formation section is per- 
formed based on the parameters specified at the param- 
eter specifying section. 

55 [0033] According to another aspect of the present in- 
vention, it is preferable that at least two of the input sec- 
tion, formation section, ai^d server section are connect- 
ed via a network. ^ 



3 



BNSDOCID: <EP 1 153638A1_I_> 



5 



EP 1 153 638 A1 



6 



[0034] According to another aspect of the present in- 
vention, it is preferable that the apparatus further com- 
prises a display section for displaying an image of the 
three-dimensional shape data seen from an arbitrary 
point of view. 

[0035] According to another aspect of the present in- 
vention, it is preferable that the apparatus further com- 
prises a coloring section for coloring the real three-di- 
mensional model based on the three-dimensional 
shape data. 

[0036] According to another aspect of the present in- 
vention, it is preferable that the display section is a dis- 
play device capable of th ree-dimensional display includ- 
ing stereo-scopic display. 

[0037] In order to achieve the object mentioned 
above, accordingto one aspect of the present invention, 
there is provided a three-dimensional modeling appara- 
tus for supplying a three-dimensional model of a target 
to a customer, comprising a data input seclion for input- 
ting three-dimensional shape data of the target and a 
formation section for forming a real three-dimensional 
model based on the obtained three-dimensional shape 
data, wherein the real three-dimensional model com- 
prises a plurality of parts, and a portion of the parts are 
selected from among a plurality of pre-provided compo- 
nents based on the measured three-dimensional shape 
data of the target. 

[0038] According to another aspect of the present in- 
vention, it is preferable that the selected component is 
further processed. 

[0039] According to another aspect of the present in- 
vention, it is preferable that the real three-dimensional 
model is constructed from at least three parts, the first 
and second parts are constructed from an identical first 
material, the third part is constructed from a material dif- 
ferent from the first material, three-dimensional shape 
data which correspond to at least the first part is input 
at the input section, the first part is formed at the forma- 
tion section based on the input three-dimensional shape 
data, and the formed first and second parts and second 
and third parts are respectively integrated by joining re- 
spective parts together. 

[0040] In order to achieve the object described above, 
according to another aspect of the present invention, 
there is provided a three-dimensional modeling appara- 
tus for supplying a three-dimensional model of a target 
to a customer, the apparatus comprising an input sec- 
tion for inputting three-dimensional shape data of the 
target; a recording section for recording a plurality of 
three-dimensional shape data which are input from the 
input section; an identification number input section for 
inputting an identification number which is used for dis- 
playing the recorded three-dimensional shape data; an 
identification number judging section for judging the in- 
put identification number; and a data outputting section 
for outputting the recorded three-dimensional shape da- 
ta. 

[0041 ] According to another aspect of the present in- 



vention, it is preferable that the data can be displayed 
from viewpoints within a predetermined range for the 
display process of the three-dimensional shape data at 
the data outputting section. 

s [0042] According to another aspect of the present in- 
vention, it is preferable that there be a plurality of the 
predetermined points of view, that, based on the identi- 
fication number, one point of view is selected from 
among the plurality of predetermined points of view, and 

10 that the three-dimensional shape data is displayed at 
the data outputting section based on the selected point 
of view. 

[0043] According to another aspect of the present in- 
vention, it is preferable that the apparatus further com- 
*5 prises a modifying section for reading and modifying the 
three-dimensional shape data recorded at the recording 
section, and tor recording the modified data to the re- 
cording section. 

[0044] According to another aspect of the present in- 
20 vention, it is preferable that at least two of the input sec- 
tion, recording section, identification number input sec- 
tion, identification number judging section, data output- 
ting section, and modifying section are connected via a 
network. 

25 [0045] According to another aspect of the present in- 
vention, it is preferable that the data outputting section 
is a display device capable of three-dimensional display 
including stereo-scopic display. 

[0046] According to another aspect of the present in- 

30 vention, it is preferable that the target is a person, voice 
of the target person is also input at the input section, the 
voice is recorded at the recording section in addition to 
the three-dimensional shape data, and the data output- 
ting section is capable of outputting the three-dimen- 

35 sional shape data and/or the voice. 

. [0047] According to another aspect of the present in- 
vention, it is preferable that a plurality of face expres- 
sions are input at the input section for one person, the 
recording section records the plurality of face expres- 

40 sions, and the display/outputting section is capable of 
sequentially displaying the plurality of face expressions 
in response to directions by the customer. 
[0048] In orderto achieve the object described above, 
according to another aspect of the present invention, 

45 there is provided a three-dimensional modeling appara- 
tus for supplying a three-dimensional model of a target 
to a customer, the apparatus comprising an input sec- 
tion for inputting three-dimensional shape data of the 
target; a database section for registering the three-di- 

50 mensional shape data input from the input section and 
its property information; and a display section for dis- 
playing the registered three-dimensional shape data. 
[0049] According to another aspect of the present in- 
vention, it is preferable that the apparatus further com- 

55 prises a modifying section for reading and modifying the 
three-dimensional shape data registered at the data- 
base section, and for re-registering the modified data to 
the database section. 
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[0050] According to another aspect of the present in- 
vention, it is preferable that a thumbnail image of each 
of three-dimensional shape data is registered at the da- 
tabase section. 

[0051] According to another aspect of the present in- 
vention, it is preferable that the thumbnail image is pro- 
duced by assigning a point of view for the three-dimen- 
sional shape data and obtaining a two-dimensional im- 
age seen from the assigned point of view. 
[0052] According to another aspect of the present in- 
vention, it is preferable that the thumbnail image is pro- 
duced by assigning a plurality of points of view for the 
three-dimensional shape data and obtaining a series of 
two-dimensional images seen from these points of view. 
[0053] According to another aspect of the present in- 
vention, it is preferable that the thumbnail image is pro- 
duced by selecting one of the two-dimensional images 
that were taken when inputting the three-dimensional 
shape data, and reducing the selected two-dimensional 
image. 

[0054] According to another aspect of the present in- 
vention, it is preferable that the thumbnail image is a 
reduced three-dimensional shape data obtained by re- 
ducing the data in the three-dimensional shape data. 
[0055] According to another aspect of the present in- 
vention, it is preferable that for each of the data regis- 
tered at the database section the number of accesses 
are additionally registered. 

[0056] According to another aspect of the present in- 
vention, it is preferable that the apparatus further com- 
prises a charging device, wherein charge process for 
charge targets registered in each database is executed 
based on the number of accesses. 
[0057] According to another aspect of the present in- 
vention, it is preferable that the apparatus further com- 
prises a refund device, wherein refund process for re- 
fund targets registered in each database is executed 
based on the number of accesses. 
[0058] According to another aspect of the present in- 
vention, it is preferable that the display section is a dis- 
play device capable of th ree-dimensional display includ- 
ing stereo-scopic display. 

[0059] According to another aspect of the present in- 
vention, it is preferable that the apparatus further com- 
prises a motion input section for inputting motion infor- 
mation of the three-dimensional shape data input from 
the input section; a dividing section for dividing the 
three-dimensional shape data input from the input sec- 
tion into a plurality of portions; and a motion recording 
section for linking and recording the motion information 
and the plurality of divided portions. 
[0060] In order to achieve the object described above, 
according to one aspect of the present invention, there 
is provided a three-dimensional model preparation 
method for supplying a three-dimensional model of a 
target to a customer via a three-dimensional modeling 
apparatus , the method comprising the steps of an input 
step for inputting three-dimensional shape data of the 



target; a formation step for forming a real three-dimen- 
sional model based on the obtained three-dimensional 
shape data; and a recording medium issuing step for 
issuing a recording medium in which information can be 

5 written to or read from, when the three-dimensional 
shape data is input; wherein the recording medium can 
be read by the three-dimensional modeling apparatus 
and/or a third-party apparatus other than the three-di- 
mensional modeling apparatus . 

10 [0061] According to another aspect of the present in- 
vention, it is preferable that the recording medium is ca- 
pable of recording data identification information. 
[0062] According to another aspect of the present in- 
vention, it is preferable that the recording medium allows 

15 new information to be written by the three-dimensional 
modeling apparatus and/or an third-party apparatus oth- 
er than the three-dimensional modeling apparatus. 
[0063] According to another aspect of the present in- 
vention,^ is preferable that the method further compris- 

20 es a recording medium information re-input step for re- 
reading a recording medium on which new information 
has been written. 

[0064] According to another aspect of the present it is 
preferable that the formation parameters are changed 

25 based on the information input at the recording medium 
information re-input step and a real object model is 
formed from the changed formation parameters. 
[0065] According to another aspect of the present in- 
vention, it is preferable that the input three-dimensional 

30 shape data is recorded on the recording medium. 

[0066] According to another aspect of the present in- 
vention, it is preferable that the third-party apparatus is 
an entertainment device which reads the three-dimen- 
sional shape data recorded on the recording medium 

35 and uses an entertainment program of the entertain- 
ment device. 

[0067] In order to achieve the object described above, 
according to another aspect of the present invention, 
there is provided a three-dimensional model preparation 

40 method for supplying a three-dimensional model of a 
target to a customer via a three-dimensional modeling 
apparatus , the method comprising the steps of an input 
step for inputting three-dimensional shape data of the 
target; a formation step for forming a real three-dimen- 

45 sional model based on the obtained three-dimensional 
shape data; a communication step for communicating 
with the three-dimensional modeling apparatus and/or 
a third-party apparatus other than the three-dimensional 
modeling apparatus via a network; and a formation proc- 

50 ess progress information transmitting step for transmit- 
ting the progress information of the formation process 
to the other apparatus. 

[0068] According to another aspect of the present in- 
vention, it is preferable that the formation process 
55 progress information at the formation step includes for- 
mation completing time information and the time infor- 
mation is displayed at a display section provided on the 
apparatus. 



5 



BNSDOCID: <EP. 



1153638A1J_> 



9 



EP1 153 638 A1 



10 



[0069] According to another aspect of the present in- 
vention, it is preferable that at the communication step, 
the information at the apparatus is received and param- 
eters for forming the real three-dimensional model are 
changed based on the information, and that at the for- 
mation step, a real three-dimensional model is formed 
based on the changed parameters. 
[0070] According to another aspect of the present in- 
vention, it is preferable that the three-dimensional shape 
data is transmitted to the apparatus through the com- 
munication step. 

[0071 ] According to another aspect of the present in- 
vention, it is preferable that the apparatus is an enter- 
tainment device and the transmitted three-dimensional 
model shape data is used by the entertainment device. 
[0072] In orderto achieve the object described above, 
according to another aspect of the present invention, 
there is provided a three-dimensional model preparation 
method for supplying a three-dimensional model of a 
target to a customer via a three-dimensional modeling 
apparatus , the method comprising the steps of an input 
step for inputting three-dimensional shape data of the 
target; a formation step for forming a real three-dimen- 
sional model based on the obtained three-dimensional 
shape data; a contacting means input step for inputting 
information contacting means for the customer; and an 
information notifying step for notifying modeling infor- 
mation to the information contacting means. 
[0073] According to another aspect of the present in- 
vention, it is preferable that the information contacting 
means for the customer is a cellular phone, the phone 
number for the cellular phone is input at the contacting 
means input step, and the modeling information is the 
progress information of formation process at the forma- 
tion step. 

[0074] According to another aspect of the present in- 
vention, it is preferable that the information contacting 
means for the customer is an electric mail, the mail ad- 
dress for the electric mail is input at the contacting 
means input step, and the modeling information is for- 
mation process completion information. 
[0075] According to another aspect of the present in- 
vention, it is preferable that the modeling information is, 
in addition to being the formation completion informa- 
tion, identification information for accessing a server 
which stores the three-dimensional shape data. 
[0076] According to another aspect of the present in- 
vention, it is preferable that the modeling information is 
three-dimensional shape data. 

[0077] According to another aspect of the present in- 
vention, it is preferable that the method further compris- 
es the steps of an estimated completion time display 
step for displaying the shortest estimated time for the 
formation completion and a formation completion time 
specifying step for specifying a time after the shortest 
formation completion time, wherein a schedule for the 
formation process is determined to accommodate the 
completion time indicated at the formation completion 



time specifying step by the customer. 
[0078] According to another aspect of the present in- 
vention, it is preferable that the method further compris- 
es a process start receiving step for receiving start time 
5 for the formation process, wherein the formation proc- 
ess is started according to the start time indicated by the 
customer. 

[0079] According to another aspect of the present in- 
vention, it is preferable that the method further compris- 
es the steps of an address means input step for inputting 
delivery destination address for the real three-dimen- 
sional model and a delivery destination printing step for 
printing the delivery destination address. 
[0080] According to another aspect of the present in- 
vention, it is preferable that the method further compris- 
es a formation section selecting step for selecting a suit- 
able formation section based on the input address from 
among a plurality of forming sections connected via a 
network, wherein the three-dimensional shape data is 
transmitted to the selected formation section. 
[0081 ] According to another aspect of the present in- 
vention, it is preferable that the method further compris- 
es a communication step for communicating with a da- 
tabase section via a network, wherein the three-dimen- 
sional shape data is recorded at the database section, 
[0082] According to another aspect of the present in- 
vention, it is preferable that the method further compris- 
es athree-dirnensionalshape data synthesizing step for 
reading three-dimensional shape data stored in the da- 
tabase section and for generating new three-dimension- 
al shape data obtained by integrating a plurality of three- 
dimensional shape data. 

[0083] According to another aspect of the present in- 
vention, it is preferable that the method further compris- 
es a managing step for managing the three-dimensional 
shape data input at the input step, wherein one forma- 
tion section is selected from among a plurality of forma- 
tion sections at the managing step for assigning the 
three-dimensional shape data input at the input step. 
[0084] According to another aspect of the present in- 
vention, it is preferable that the selection of the forma- 
tion section is performed based on the operation condi- 
tions at the plurality of formation sections. 
[0085] According to another aspect of the present in- 
vention, it is preferable that the method further compris- 
es a parameter specifying step for the customer to spec- 
ify the material or size of the object to be formed, where- 
in the selection of the formation section is performed 
based on the parameters specified at the parameter 
specifying step. 

[0086] According to another aspect of the present in- 
vention, it is preferable that data transmission/reception 
between at least a pair of the input step, formation step, 
and managing step are performed via a network. 
[0087] According to another aspect of the present in- 
vention, it is preferable that the method further compris- 
es a display step for displaying an image of the three- 
dimensional shape data seen from an arbitrary point of 
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view. 

[0088] According to another aspect of the present in- 
vention, it is preferable that the method further compris- 
es a coloring step for coloring the real three-dimensional 
model based on the three-dimensional shape data. 
[0089] In order to achieve the object described above, 
according to another aspect of the present invention, 
there is provided a three-dimensional modeling method 
for supplying a three-dimensional model of a target to a 
customer, the method comprising the steps of a data in- 
put step for inputting three-dimensional shape data of 
the target and a formation step for forming a real three- 
dimensional model based on the obtained three-dimen- 
sional shape data, wherein the real three-dimensional 
model comprises a plurality of parts, and a portion of the 
parts is selected from among a plurality of pre-prbvided 
components based on the measured three-dimensional 
shape data of the target. 

[0090] According to another aspect of the present in- 
vention, it is preferable that the selected component is 
further processed. 

[0091] According to another aspect of the present in- 
vention, it is preferable that the real three-dimensional 
model is constructed from at least three parts, the first 
and second parts are constructed from an identical first 
material, the third part is constructed from a material dif- 
ferent from the first material, three-dimensional shape 
data which correspond to at least the first part is input 
at the input step, the first part is formed at the formation 
step based on the input three-dimensional shape data, 
and the formed first and second parts and second and 
third parts are respectively integrated by joining respec- 
tive parts together. 

[0092] In orderto achieve the object described above, 
according to another aspect of the present invention, 
there is provided a three-dimensional modeling method 
for supplying a three-dimensional model of a target to a 
customer, comprising the steps of an input step for in- 
putting three-dimensional shape data of the target; a re- 
cording step for recording a plurality of three-dimension- 
al shape data which are input at the input step; an iden- 
tification number input step for inputting an identification 
number which is used for displaying the recorded three- 
dimensional shape data; an identif ication number judg- 
ing step forjudging the input identification number; and 
a data display/outputting step for displaying/outputting 
the recorded three-dimensional shape data. 
[0093] According to another aspect of the present in- 
vention, it is preferable that the data can be displayed 
from viewpoints within a predetermined range at the dis- 
play/outputting step of the three-dimensional shape da- 
ta. 

[0094] According to another aspect of the present in- 
vention, it is preferable that there are a plurality of the 
predetermined points of view, a point of view is selected 
based on the identification number, and the three-di- 
mensional shape data is displayed at the data display/ 
outputting step based on the selected point of view. 



[0095] According to another aspect of the present in- 
vention, it is preferable that the method further compris- 
es a modifying steo for reading and modifying the three- 
dimensional shape data recorded at the recording sec- 
5 tion, and for writing the modified data to the recording 
section. 

[0096] Accorcing to another aspect of the present in- 
vention, it is pref erable that data transmission/reception 
between at least a pair of the input step, recording step, 
10 identification number input step, identification number 
judging step, data display/outputting step, and modify- 
ing step are performed via a network. 
[0097] According to another aspect of the present in- 
vention, it is ^referable that the target is a person, voice 
15 of the target person is also input at the input step, the 
voice is recorded at the recording step in addition to the 
three-dimensional shape data, and the voice is also dis- 
played with the three-dimensional shape data at the da- 
ta display/outputting step. 
20 [0098] According to another aspect of the present in- 
vention, it is preferable that a plurality of face expres- 
sions arc input at the input step for one person, the plu- 
rality of ^ace expressions are recorded at the recording 
step, and the plurality of face expressions are sequen- 
25 tially d:splayed at the display/outputting step in re- 
sponse to directions by the customer. 
[0099] In order to achieve the object mentioned 
above, according to one aspect of the present invention, 
there ss provided a three-dimensional modeling method 
30 for supplying a three-dimensional model of a target to a 
customer, the method comprising the steps of an input 
step for inputting three-dimensional shape data of the 
target; a data managing step for registering the three- 
dimensional shape data input at the input step and its 
35 property information; and a display step for displaying 
tre registered three-dimensional shape data. 
[0100] According to another aspect of the present in- 
vention, it is preferable that the method further compris- 
es the step of a modifying step for reading and modifying 
40 the three-dimensional shape data registered at the data 
managing step, and for re- registering the modified data. 
[0101] According to another aspect of the present in- 
vention, it is preferable that a thumbnail image of each 
of three-dimensional shape data is also registered at the 
4 -' data managing step. 

[0102] According to another aspect of the present in- 
vention, it is preferable that the thumbnail image is pro- 
duced by assigning a point of view for the three-dimen- 
sional shape data and obtaining a two-dimensional im- 
50 age seen from the assigned point of view. 

[0103] According to another aspect of the present in- 
vention, it is preferable that the thumbnail image is pro- 
duced by selecting one of the two-dimensional images 
that were taken when inputting the three-dimensional 
55 shape data, and reducing the selected two-dimensional 
image. 

[01 04] According to another aspect of the present in- 
vention, it is preferable that for each of the registered 
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Fig. 1 7 is a diagram showing coloring process using 
a stamp. 

Fig. 18 is a diagram showing coloring process by 
three-axis driving an ink jet nozzle. 
5 Fig. 1 9 is a diagram showing an example where an 

ink jet nozzle and a milling head are placed togeth- 
er. 

Fig. 20 is a diagram showing a structure of a three- 
dimensional data generating apparatus according 
10 to another embodiment of the present invention. 

Fig. 21 (a) and 21 (b) are diagrams for explaining the 
positioning of irradiation sources. 
Fig. 22(a) and 22(b) are diagrams for explaining a 
laser ray irradiating device. 
*5 Fig. 23(a) and 23(b) are diagrams for explaining the 
positioning of irradiation sources. 
Fig. 24 is a diagram showing a structure of a three- 
dimensional data generating apparatus according 
to yet another embodiment of the present invention. 
20 Fig. 25 is a diagram showing the flow of data input. 

Fig. 26 is a diagram showing an exemplary struc- 
ture of a three-dimensional modeling apparatus ac- 
cording to still another embodiment of the present 
invention. 

25 Fig. 27 is a diagram showing a method for incorpo- 

rating the real three-dimensional model. 
Fig. 28 is a diagram showing an example of the con- 
tents of the database. 

30 Best Mode for Carrying Out the Invention 
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data the number of accesses br additionally registered. 
[0105] According to anothc r expect of the present in- 
vention., it is preferable thattho n 2thod further compris- 
es a charging step, wherein a chage target is registered 
at the data managing step, and charge process for the 
recorded charge target is executed based on the 
number of accesses. 

[0106] According to another aspect of the present in- 
vention, it is preferable that the metr od further compris- 
es a refund step, wherein a refund target is registered 
at the data managing step and refund process for the 
registered refund target is executed based on the 
number of accesses. 

[0107] According to another asp^ci f the present in- 
vention, it is preferable that the met no-, urther compris- 
es the steps of a motion input step or nputting motion 
information of the three-dimension^ snape data input 
at the input step; a dividing step for div.ding the three- 
dimensional shape data input at the inoui step into a plu- 
rality of portions; and a motion record;. :g : [ep for linking 
and recording the motion information an^ the plurality 
of divided portions. 

Brief Description of Drawings 

[0108] 

Fig. 1 is a block diagram showing a si uc ure of the 
present invention configured according t: one pre- 
ferred embodiment. 

Fig. 2 is a conceptual diagram showinc a structure 
of the present invention in the preferred embodi- 
ment. 

Fig. 3 is a diagram showing calibration. 

Fig. 4 is a diagram showing a structure for ore ecting 

a random pattern. 

Fig. 5 is a diagram exemplifying a randon op;tern. 
Fig. 6 is a diagram showing usage of a temp, -te. 
Fig. 7 is a diagram showing a structure of a orning 
mold. 

Fig. 8 is a diagram showing another exemt ary 
structure of a forming mold. 

Fig. 9 is a diagram showing yet another exernolory 
structure of a forming mold: 

Fig. 1 0 is a flowchart showing the coloring pre cess 
using a resist. 

Fig. 11 is a flowchart showing the coloring process 
using a photosensitive agent. 

Fig. 1 2 is a diagram showing a structure for pattern 
projection. 

Fig. 13 is a diagram showing another exemplary 
structure for pattern projection. 
Fig. 1 4 is a flowchart for the coloring process by re 
moving unnecessary parts. 

Fig. 15 is a diagram showing the coloring process 
by removing unnecessary parts. 
Fig. 16 is a diagram showing the coloring process 
using a heat-shrinkable film. 



[0109] Preferred embodiments of the present inven- 
tion will now be described referring to the drawings. 
[0110] Figs. 1 and 2 show a structure of a real three- 
dimensional model generating apparatus according to 
a first preferred embodiment of the present invention. 
[0111] A data inputting section 10 captures a target 
such as a person or the face of a person, and produces 
an image data. The data inputting section 1 0 includes a 
plurality of color CCD cameras, as shown in Fig. 2 : and 
obtains color image data of the target at a plurality of 
positions. 

[01 12] The color image data including the color infor- 
mation obtained by the data inputting section 1 0 is then 
input to a modeling section 12. The modeling section 12 
includes a personal computer or the like, and performs 
three-dimensional modeling based on the plurality of im- 
age data of the target captured at the plurality of posi- 
tions, to thereby obtain three-dimensional shape data of 
the target. 

[0113] The three-dimensional shape data includes 
the color data of the target. For the three-dimensional 
modeling, any method can be used such as, for exam- 
ple; the method disclosed in Japanese Patent Laid- 
Open Publication No. Hei 10-124704. 
[0114] The obtained three-dimensional shape data is 
input to a forming section 14. The forming section 14 is, 
for example, a milling drill which can move with respect 
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to three axis such as xyz, and mills a work to produce a 
real three-dimensional model based on the input three- 
dimensional shape data. 

[0115] The obtained real three-dimensional model is 
then supplied to a coloring process section 16. The 
coloring process section is for coloring the real three- 
dimensional model, and when the target is a person, 
colors at least the eyes and the mouth. 
[0116] As described, in this embodiment, the image 
data is obtained by capturing the target using cameras. 
Thus, it is safe even when the target is a person. More- 
over, color data is obtained for coloring at the coloring 
process section. Because of this, a real three-dimen- 
sional model can be produced which includes color in- 
formation, and thus, contains unique characteristics. 
[01 17] In the following description, the term "three-di- 
mensional shape data" refers to data on the three-di- 
mensional shape information of the target, and may in- 
clude the color information. The three-dimensional 
shape data is obtained by inputting an image at the data 
inputting section and performing a modeling process at 
the modeling section. When the data inputting section 
is a three-dimensional shape measurement device, the 
three-dimensional shape data may be obtained at the 
data inputting section. Moreover, data which is obtained 
by changing, modifying, integrating, and/or processing 
the obtained three-dimensional shape data is also re- 
ferred to as the three-dimensional shape data. 
[0118] The term "real three-dimensional model", on 
the other hand, refers to a three-dimensional object pro- 
duced at the forming section based on the obtained 
three-dimensional shape data. The "real three-dimen- 
sional model" also includes the real three-dimensional 
model as defined above, colored at the coloring process 
section. 

[0119] The term "three-dimensional data" is used in- 
terchangeably with the term "three-dimensional shape 
data" (which may include the color information). A term 
"pseudo-three-dimensional data", on the other hand, re- 
fers to data which does not contain the three-dimension- 
al shape information of the target but which can display 
the target in virtual three-dimension. One example of 
pseudo-three-dimensional data is QuickTime VR data. 
The term "three-dimensional model" refers to both a real 
three-dimensional model and to three-dimensional data 
as defined above. "Structure of the Data Inputting Sec- 
tion and the Modeling Section" 

[0120] The data inputting section 10 uses cameras to 
capture a target. Generally, in order to obtain three-di- 
mensional shape data, a plurality of image data from a 
plurality of directions are necessary for each immovable 
target. Therefore, it is preferable to fix a plurality of cam- 
eras at predetermined positions to simultaneously cap- 
ture the target. However, it is also possible to movably 
mount a camera on a structure such as a rail and obtain 
a plurality of image data by moving the camera. 
[0121] The absolute positions of the plurality of the 
fixed cameras must be precisely determined, but in gen- 



eral this may be difficult with the mechanical means. 
Therefore, it is preferable to perform calibration of the 
camera positions by placing a reference object with a 
predetermined pattern in the camera view and calibrate 

5 the camera position based on the image data of the ref- 
erence object, as shown in Fig. 3. In this manner, the 
mounting of the cameras itself can be simplified and, at 
the same time, their positions can be precisely detected. 
It is preferable to perform the calibration periodically, e. 

10 g., on an everyday basis, or on an every week basis. 
[0122] In order to precisely obtain three-dimensional 
shape data or the like of a target, each of the sections 
of the target must be seen differently from a plurality of 
directions. Therefore, it is preferable to project a prede- 

15 termined pattern on the target using a projector as 
shown in Fig. 4. In this manner, precise three-dimen- 
sional shape data can be obtained even for sections with 
uniform colors or where there is little change in color. 
The pattern can be removed from the color data by data 

20 processing or, alternatively, the color data can be ob- 
tained from the image data without the pattern project- 
ed. A random pattern as shown in Fig. 5, for example, 
can be used as the projection pattern. 
[01 23] When obtaining three-dimensional shape data 

25 using such a random pattern, by obtaining depth data 
by a stereo method using a plurality of cameras, a high 
precision modeling is possible even for a recess section. 
[0124] Forthe camera, it is preferable to use a camera 
with a wide-angle lens. By using such cameras, the 

30 cameras can be placed in relatively small spaces. 

[01 25] The data inputting section 1 0 captures the tar- 
get by the cameras and extracts the target area in the 
image. It is therefore preferable to make the background 
a constant color to facilitate extraction of the target area 

35 in the image. 

[0126] For example, it is preferable to provide a room 
with the four walls colored in one color and to place the 
target at the center of the room. In particular, when the 
target is a person, it is preferable to place a chair at the 

40 center of the room and let the person (target) sit on the 
chair. When the target is a person, the image captured 
from the back is usually of less importance. 
[0127] Therefore, it is preferable to provide an en- 
trance at one wall of the room, have the person sit facing 

45 the entrance, and to unify the color of the walls other 
than the wall with the entrance. It is also preferable to 
omit the image data from the back when the target is a 
person. 

[01 28] Furthermore, when the target has a color iden- 
so tical to the background color, extraction can not be per- 
formed correctly. Therefore, it is preferable to configure 
the room so that the background color (the wall color of 
the room) is changeable. For example, by using a pro- 
jector, light of a predetermined color can be projected 
55 onto the wall, to thereby change its color. 

[0129] It is also preferable to use at least two colors 
for the background color, obtain image data with two 
types of background color, and extract the target area 
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by the union of the extraction results from the two back- 
ground colors. In this manner, precise extraction of the 
target area can be accomplished for any target of any 
color. 

[0130] It is also preferable to provide lighting appara- 5 
tuses at the comers of the room for uniform lighting ef- 
fect on the target and light up the room so that there will 
be no shadow on the target. n 

Structure of the Forming Section" 

[0131] The forming section 14 produces a real three- 
dimensional model based on the three-dimensional 
shape data. As the forming section 14, a milling drill ca- 
pable of three-dimensional processing can be used, as 
described before. When the target is the face of a per- 
son, a process with only one drill axis direction is also 
possible. However, in order to precisely process the 
nostril or the like, it is preferable to use a processing 
device which can also perform rotation of the drill axis 
direction. 

[0132] The material to be used as the basis for the 
real three-dimensional model includes a scantling and 
a round bar. However, when the target is known to some 
extent, for example, when the target is a human face, it 
is preferable to prepare a template (a prototype of the 
work to be processed) which resembles the shape of 
the target and process the template. In this manner, the 
time required for the process can be reduced. Moreover, 
various types of templates can be prepared, as shown 
in Fig. 6, and a template closely resembling the obtained 
three-dimensional shape data can be selected for proc- 
ess. 

[0133] In the example shown in Fig. 6, a template for 
a round face and a template for thin face are separately 
prepared, and a suitable template can be selected de- 
pending on the face of the target and processed. In this 
manner, processing can be facilitated and the process- 
ing time can be further reduced. Moreover, by using 
such a template, the amount of waste generated during 
milling can also be reduced. Furthermore, it is also pos- 
sible to modify the obtained three-dimensional shape 
data by morphing so that the data more closely resem- 
bles the shape of the selected template and to process 
the template, resulting in further reduction of the 
processing time. 

[0134] For the portion corresponding to the hair, it is 
possible to use a pre-provided template without 
processing. Specifically, the hair is not usually as impor- 
tant, and it is likely that atempiatechosen from few types 
of pre-provided templates is sufficient. Therefore, by 
preparing few types of templates for the hair portion, it 
is possible to select and use from among the prepared 
templates. It is also possible to separately prepare the 
templates for the hair portion and the templates for the 
face portion. 

[0135] In this case, the hair portion can be colored in 
black. In this manner, coloring process for the hair por- 



tion can be omitted. Even when coloring process must 
be performed, by separately coloring the hair portion, 
the overall coloring procedure can be simplified. It is 
preferable that the hair portion is formed so that it is ap- 
plied to the face portion by covering on the upper section 
of the face, similar to a wig. It is also possible to form 
the hair portion as a wig with a thread as the hair. 
[0136] It is also possible to generate a forming mold 
and produce a real three-dimensional model using the 
mold. Fig. 7 shows a forming mold using a flexible film 
30 and a pin assembly 32. The pin assembly 32 is con- 
structed from a plurality of pins 34, each of which is mov- 
able and fixable. As shown in the figure, by determining 
the positions of one or more of pins 36 by an actuator 
and applying the pin(s) 36 via the flexible film 30, the 
positions for each of the pins 34 of the pin assembly 32 
can be set. Once the shape of the pin assembly 32 is 
determined, the shape of the flexible film 30 which cov- 
ers the pin assembly is also determined. 
[0137] It is also possible to create a forming mold by 
constructing an object with the pin assembly 32, in which 
the positions of pins 34 are fixed, and the flexible film 
30, and generate a real three-dimensional model by fill- 
ing a material into the forming mold and solidifying the 
material. By forming a mold in this manner, a plurality of 
real three-dimensional models can be easily produced. 
[0138] For example, by using a heat-resistive film as 
the flexible film 30, a thermosetting resin can be used 
as the material for formation by heating. 
[0139] By applying a reset plate 38 from the opposite 
side of the pin assembly 32, the pins 34 of the pin as- 
sembly 32 pushed by the pin 36 can be returned to their 
original positions and be ready for the next formation. 
When the shape of the target is known to some extent, 
such as when the target is a human face, it is also pref- 
erable to set the shape of the reset plate 38 into a shape 
corresponding to a general shape corresponding to the 
target in order to set the default shape of the pin assem- 
bly 32 close to the final shape of the target. 
[0140] The pins 34 of the pin assembly 34 need not 
be facing the same direction. It is also possible to set an 
axial direction for each pin and place the pins according 
to the set directions : in order to support a curved shape. 
[0141] In the example shown in Fig. 7, the pins are 
passively moved. However, it is also possible to provide 
an actuator 40 corresponding to each pin 34 and control 
the shape of the pin assembly 32 by driving a plurality 
of the actuators 40. In this case, it is also preferable to 
provide a flexible film 30 on top of the pins 34. 
[0142] Fig. 9 shows an example where each of the 
pins 34 are provided in a shape corresponding to a 
sphere. By arranging the pins 34 in this manner, each 
pin 34 extends radially outward and actuators 40 are 
placed thereon. By doing so, the space for placing each 
actuator 40 can be broadened and, at the same time, 
the density of the head of the pins 34 can be increased. 
[0143] The above configuration is further preferable 
because a human face resembles a sphere. The shape 
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of the group of heads of the pins 34 need not be shaped 
into a perfect sphere, and can be any suitable arrange- 
ment based on the shape of the target. 
[0144] With this arrangement, it is also possible to in- 
dividually determine the positions of the heads of the 
pins 34 by individually driving the actuators 40, in order 
to shape the flexible film 30 supported by the pins 34 
into a desired shape. Thus, formation of a real three- 
dimensional model with a desired shape can be 
achieved. 

[0145] When forming, instead of using the three-di- 
mensional shape data in its original form, it is also pos- 
sible to shrink the scale of the three-dimensional shape 
data along the depth axis and form areal three-dimen- 
sional model based on the compressed data. A relief- 
type real three-dimensional model can be produced. 
The relief-type real three-dimensional model is easier to 
form, and the milling waste due to usage of an engraving 
machine can be reduced. 

[0146] It is also preferable to extract a characteristic 
portion of the target and form a model accordingly. For 
example, by applying an edge exaggerating process on 
the three-dimensional shape data, the characteristic 
portion of the three-dimensional shape data can be ex- 
aggerated. By producing a real three-dimensional mod- 
el with the characteristic portion exaggerated, a real 
three-dimensional model which accurately capture the 
characteristics of the target can be obtained. 
[0147] It is also possible to produce the forming mold 
using clay or the like. In this case, the clay or the like 
can be recycled. " 

Structure of Coloring Process Section" 

[0148] The coloring process section 1 6 applies color- 
ing process on the real three-dimensional model ac- 
cording to the color data. Various methods exist for the 
coloring process, and these methods will be described 
below. 

[0149] First, a resist can be applied on the real three- 
dimensional model and coloring process can be per- 
formed using the resist. This method will be described 
referring to Fig. 10. A resist is applied on the surface of 
the real three-dimensional model (step S11 ). When the 
target is a human face, it is only necessary to color the 
face, and thus, in such a case, the face portion is the 
only target for the coloring process. Then, using a pre- 
determined pattern, by exposing the necessary portion 
or by thinly milling the necessary portion with a drill or 
the like, the resist on the necessary portion is partially 
removed (step S12). The partial removal is performed 
for the portion to be colored for each color. Coloring 
process for a color is then performed on the removed 
portion (step S13). 

[0150] It is then judged whether coloring process for 
all the colors are completed (step S14). If not, the pro- 
cedure returns to step S11 and performs coloring proc- 
ess for the next color. Here, by using a material that re- 



pels the paint used for coloring, selective coloring on the 
portion where the resist is removed is possible. Because 
the resist is fairly thin, it does not cause a problem even 
if the portion of the resist which is not removed is left on 

5 the model. It is preferable to cover the entire model with 
a durable protection film after all the coloring processes 
are completed. In this method, it is better to minimize 
the number of colors. Accordingly, if the target is a hu- 
man face, it is preferable to simplify the process to some 

10 extent by, for example, coloring the eyes with a black 
color and lips with a red color. 

[01 51 ] When the above described coloring method is 
employed, there is a limit on the number of colors for 
coloring the target. Therefore, there is a need for a re- 
15 ducing process to reduce the number of colors in the 
original color data included in the obtained image data. 
Preferable reduction can be accomplished by, for exam- 
ple, a process with the following steps: 

20 1 . Divide the obtained image data into regions, 

2. Obtain the average color value within one region, 

3. Compare the average color value with all the pre- 
pared available colors for each region to obtain the 
available color closest to the average color value, 

25 and 

4. Replace the color within the region by the avail- 
able color determined at step 3. 

[0152] It is also preferable to use a photosensitive 
30 agent for the coloring process. This method will be de- 
scribed with reference to Fig: 11 . First, a photosensitive 
agent is applied on the real three-dimensional model 
(step S21). Then, a pattern is projected based on the 
color data to expose the photosensitive agent (step 
35 S22). 

[0153] A fixing process is applied to the photosensi- 
tive agent to fix the color (step S23). The pattern projec- 
tion can be achieved, as shown in Fig. 12, by irradiating 
light from a light source onto a real three-dimensional 
40 model (three-dimensional object), with the photosensi- 
tive agent applied, via an projection pattern. The projec- 
tion pattern can be, for example, a transmission type liq- 
uid crystal panel. 

[0154] It is also possible to directly project a pattern 
•*5 from a CRT on the real three-dimensional model. In this 
example, the three-dimensional model is a face, and, 
thus, the exposure is performed with a single irradiation 
from the front side. In particular, by using a long focal 
length lens to ensure a sufficient focus depth even for a 
50 three-dimensional object, exposure of the entire face 
portion (hemisphere) can be performed using one pro- 
jection pattern. 

[0155] Shrinking the scale along the depth axis for a 
relief-type real three-dimensional model increases the 
55 likelihood of obtaining a uniform exposure. As a photo- 
sensitive agent, silver halides such as silver bromide, 
silver chloride, or silver iodide can be used. These pho- 
tosensitive agents are applied on the surface of the real 
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three-dimensional model, dried, and then exposed. 
[0156] Because the real three-dimensional model is 
a three-dimensional object there are cases where suf- 
ficiently uniform exposure cannot be achieved depend- 
ing on the direction of the surface. To this end, as shown 
in Fig. 1 3, it is preferable to provide a plurality of projec- 
tion patterns corresponding to the directions of surfaces 
of the real three-dimensional model and to expose the 
photosensitive agent from a plurality of directions. 
[0157] In this case, it is preferable to apply masks so 
that there will not be too much light incident on an over- 
lapping section from a plurality of projection patterns. In 
this case, it is preferable to fix the positions of the light 
source and the projection patterns, and vary the mask- 
ing based on the real three : dimensional model. 
[0158] Moreover, it is preferable to remove unneces- 
sary portions after completing all the coloring process- 
es. In other words, as shown in Fig. 14, the real three- 
dimensional model is colored (step S31), and then the 
unnecessary portions are removed by a drill or the like 
(step 532). In this manner, coloring of just the necessary 
portion can be achieved. For example, as shown in Fig. 
15, in a human head, the material for the portion above 
the mouth can have a black color and the material for 
the portion below the mouth can have a red color. A skin 
colored paint is then applied to the entire head. By mill- 
ing and removing the paints on the eyes, hair, and 
mouth, black eyes, black hair, and red lips can be 
formed. 

[0159] As shown in Fig. 16, it is also preferable to print 
a pattern on a heat-sh linkable film and to affix the film 
on the real three-dimensional model. In this case, a 
shrinking ratio must be calculated before printing, and 
the color must be thinned in portions with a high shrink- 
ing ratio. Accurate color after shrinkage can be obtained 
in this manner. Although when film is expanded, there 
are problems such as color division, shrinkage does not 
produce such a problem, and, therefore, thicker coloring 
is enabled. 

[0160] As the heat-shrinkable film, it is preferable to 
apply a thin coat of a water soluble polymer such as PVA 
(polyvinyl alcohol) on a film of polyvinyl chloride or flu- 
oride resin, to have a printing capability (print). It is also 
possible to use an expandable cloth instead of the film. 
[01 61] It is also possible to employ a coloring scheme 
as described in the following. 

[0162] For a human face, the main characteristic 
points lie in the eyes and mouth. Thus, it may be pref- 
erable to color only the eyes and mouth. In many cases, 
the positions of these items, rather than the shape, rep- 
resent the characteristic point. Thus, as shown in Fig. 
17, it is preferable to provide two stamps for eyes and 
one stamp for the mouth and color the real three-dimen- 
sional model by pressing these stamps against the sur- 
face. In this case, the stamps for the eyes are movable 
in the horizontal direction of the face and the stamp for 
the mouth is movable in the vertical direction of the face, 
allowing for adjustment of the stamps. By forming the 



stamp by a material which can be deformed, such as, 
for example, a sponge, the surface can easily be colored 
even when the surface is not completely flat. A sponge- 
like material is also preferable because it is easy to con- 

5 tain the coloring agent in the material. 

[0163] It is also preferable to prepare a plurality of 
shapes for the eyes and mouth , select those most close- 
ly resembling the target, and use these to color the se- 
lected shape. It is also possible to drive a plurality of 

10 brushes for each color hased on the data for the coloring 
process. 

[0164] As shown in Fig. 1 8, it is also preferable to drive 
an ink jet nozzle with three axes (rotation of the real 
three-dimensional model, and movement of the nozzle 
is in the y and z directions). In this manner, a predeter- 
mined color can be applied to a predetermined location. 
As shown in Fig. 19, it is also possible to provide both 
an ink jet nozzle and a milling head with a forming drill 
on single component. By employing such a configura- 
te tion, formation and coloring processes can be per- 
formed almost simultaneously, and there is a further ad- 
vantage that only one control mechanism is required. 
When coloring process is performed afterformation, the 
components can be driven based on the identical data 
25 for identical movement, and, thus, the coloring process 
can be efficiently performed. 

[0165] It is also possible to color the real three-dimen- 
sional model by an impact dot method using a wire dot 
and an ink ribbon. 

30 [01 66] Moreover, it is also preferable, during the color- 
ing procedure, to use a contact pin sensor to detect the 
surface position of the real three-dimensional model and 
to color based on the detected position. In this manner, 
a precise coloring process can be performed, even 

35 when the formation precision is insufficient. 

[01 67] It is also possible to transfer using a color proc- 
essed film. 

[01 68] It is also preferable to layer and bond a plurality 
of pieces of paper based on the obtained three-dimen- 

40 sional shape data to form a three-dimensional object. In 
this case, the target shape is first divided by a plurality 
of parallel planes corresponding to each of the plurality 
of sheets of paperto be layered and bonded. For exam- 
ple, such a plane can be a plane perpendicular to the y 

45 axis (vertical axis) assigned to the three-dimensional 
shape data. 

[01 69] After printing the color on the cross section be- 
tween each plane and the target shape on each piece 
of paper, the cross sections of these pieces of paper are 
50 cut, and the pieces are overlapped. A colored three-di- 
mensional object can be formed in this manner. 

"Variations" 

55 [0170] It may also be preferable to display the three- 
dimensional shape, including the color, on a display 
when the three-dimensional shape data is obtained in 
order to allow a preview of the real three-dimensional 
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model to be produced. When any number of processes 
described above are applied to compress, exaggerate 
the characteristic points, and/or limitthe colors, it is pref- 
erable to show a display which is as close to the final 
product real three-dimensional model as possible. 
[0171] It is also preferable to prepare various options 
and accessories that are different from the actual target, 
such as, for example, glasses and hair styles, and to 
permit the mounting of these options on the real three- 
dimensional model. 

[0172] Fora case of a person, the final real three-di- 
mensional model can be either the head portion only or 
the complete person. For example, when a complete 
model is desired, because the face becomes too small, 
it is preferable to form the model by emphasizing the 
face by setting the face to body ratio to 1:1. tt is also 
preferable to prepare few such variations, so that a cus- 
tomer is able to select one from among the prepared 
variations. 

[0173] When a pet such as a dog is the target it may 
be necessary to restrain the pet, such as by using a 
chain. When it is preferable to provide a chain at the 
capturing locations, by using the same color for the 
chain as the background color, extraction of the target 
from the captured data can be facilitated. 
[0174] It is preferable to integrate the present inven- 
tion as a single apparatus and place the integrated ap- 
paratus in an arcade or an amusement center. In this 
manner, a user can enter the apparatus and obtain an 
image data in the same fashion as taking a picture. After 
waiting a brief time, the user can obtain a real three- 
dimensional model doll at the takeout pocket section. A 
doll of the user can be produced in a similar manner as 
a picture or a sticker. It is also preferable, when some 
period of time is required before completion of the pro- 
duction, that a card is issued and the user be able to 
exchange the card for the produced doll. In this case, 
the doll for a user can be automatically discharged to 
the takeout section by reading the issued card by a card 
reader. 

[0175] Fig. 20 is a diagram showing a structure of a 
three-dimensional data generating apparatus of another 
embodiment of the present invention. 
[0176] A light projection section 200 projects a ran- 
dom pattern, slit pattern, or coded pattern assigned on 
a panel 201 onto a target which is a human face. The 
projected pattern light is reflected at a reflecting section 
202 and projected onto the target 203. 
[0177] An image of the target with the pattern light pro- 
jected is then captured by an image inputting section 
204 such asa CCD camera, and three-dimensional data 
of the target is produced at a modeling section 205 from 
the obtained plurality of images by any of the above de- 
scribed methods. 

[0178] When the distance between the light source of 
the light projecting section 200 and the target is small 
and the pattern is directly projected on the target, the 
target itself produces a shadow, resulting in some por- 



tion of the target not projected with the pattern. In other 
words, there will be portions which cannot be seen from 
the light source (dead angle). In a case shown in Fig. 
21(a), no pattern is projected onto a part of the neck, 

5 due to a shadow from the chin. 

[0179] Thus, the shape measurement of these por- 
tions will be difficult or inaccurate. 
[0180] One solution for preventing such possibility of 
a dead angle is to set the distance between the light 

10 source and the target sufficiently large, as shown in Fig. 
21(b). However, such a solution causes the size of the 
overall apparatus to become very large. 
[0181] Thus, a reflecting section 202 is provided to re- 
flect the projection pattern and project onto the target. 

15 [0182] Byemployingsuch amethod.the light path dis- 
tance from the light projection section 200 to the target 
can be set at a large distance while the size of the overall 
apparatus is kept from becoming large. 
[0183] It is also possible to irradiate a laser ray, other 

20 than the pattern light as described above, from the light 
projection section 200. A laser ray irradiating device 
generally has a structure such as shown in Fig. 22. 
[0184] Specifically, in Fig. 22(a), the direction of a 
beam-type laser ray is changed by a galvano mirror or 

25 a polygon mirror, in order to allow a two-dimensional 
scan. 

[0185] In Fig. 22(b), a beam-type laser ray is once 
changed into a slit light at a cylindrical lens and then the 
direction of the laser ray is changed by a galvano mirror 
30 or a polygon mirror, in order to allow a two-dimensional 
scan. 

[0186] However, even in these cases, when the dis- 
tance to the target is insufficient, there will be problems 
associated with the produced dead angle. Therefore, 
35 setting the light path distance large using a reflecting 
section is again an effective method for reducing the 
dead angle. 

[0187] The problem of dead angle, on the other hand, 
is similarly generated when an object on which the pat- 

40 tern is projected is input. For example, as shown in Fig. 
23(a), when an image of an object on which a pattern is 
projected is to be captured using a CCD camera, the 
neck portion which is covered by a shadow of the chin 
cannot be input due to the reason described above. In 

45 such a case, as shown in Fig. 23(b), the solution is to 
set the distance between the camera and the target suf- 
ficiently large : using a lens with a longer focal distance. 
However, similar to the above, there remains a problem 
that the overall size of the apparatus becomes very 

50 large. 

[0188] Thus, the camera captures an image of the 
pattern projected target as reflected at the reflecting 
section. In this manner, the distance from the target to 
the camera can be set at a large distance thereby re- 
55 ducing the dead angles, while simultaneously the size 
of the overall apparatus is kept small. 
[0189] Fig. 24 is a diagram showing a structure of a 
three-dimensional data generating apparatus according 
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to another embodiment of the present invention. 
[0190] A plurality of data inputting sections 240 for 
capturing a target from a plurality of directions and in- 
putting the image data are included. The data inputting 
sections 240 include an image inputting section 241 . As 
the image inputting section 241 , an analog or digital 
color CCD camera or the like can be used. 
[01 91 ] Some of the data inputting sections have a da- 
ta input reference parameter determining section 242 
for determining suitable values for the parameters such 
as white balance and exposure during the data input 
process. 

[01 92] The other data inputting sections, on the other 
hand , include a parameter setting section 243 for setting 
the parameters to those determined at the data input 
reference parameter determining section 242. At these 
data inputting sections, image input is performed based 
on the set parameter. 

[01 93] At the modeling section 244, three-dimension- 
al shape data of the target is calculated based on a plu- 
rality of images obtained at the data inputting sections. 
The modeling process can be, for example, the method 
disclosed in Japanese Patent Laid-Open Publication 
No. Hei 10-124704. 

[0194] The data input flow will next be described re- 
ferring to Fig. 25. 

[0195] Parameters such as white balance and expo- 
sure are automatically determined at a data inputting 
section (step S10). This can be easily accomplished by 
automatic white balance and/or automatic exposure 
functions provided on a common camera. 
[0196] The determined parameters are then notified 
to the other data inputting sections (step S11). 
[0197] At each of the data inputting sections, the no- 
tified parameters are set at the parameter setting sec- 
tion within each of the data inputting sections (step S12). 
In other words, at these data inputting sections, the pa- 
rameters are not automatically determined at the re- 
spective inputting sections, but, rather, the parameters 
are set as notified from outside. 

[0198] An image is then input at all of the data input- 
ting sections based on the set parameter (step S13). 
[0199] In this manner, by using the parameter deter- 
mined at one data inputting section as a common pa- 
rameter, and inputting the image at ail the data inputting 
sections based on the common parameter, images of 
uniform quality can be obtained. 

[0200] If, on the other hand, the parameters are indi- 
vidually set at each of the data inputting section to suit 
the individual views, different parameters will be set as 
the suitable parameter because (a) the portion of the 
target which can be seen from the inputting sections are 
different and (b) the background (portion other than the 
target) which can be seen from the inputting sections 
are different. 

[0201] Because of this, colors for the same element, 
which should, of course, be identical, actually differ for 
each of the data input sections, leading to a problem 



when one unified three-dimensional shape data is gen- 
erated. However, by determining a reference parameter 
and using the reference parameter as a common pa- 
rameter, image input is enabled which satisfies the con- 
5 dition that the color forthe same element should be iden- 
tical. 

[0202] Here, a case is shown where the reference pa- 
rameter is automatically determined at one data input- 
ting section, but the reference parameter can also be 

10 determined by an operator. There are some cases 
where an experienced operator wishes to set a suitable 
parameter value different from the value automatically 
determined by the data inputting section, based on his 
experience or when he wishes to have a special effect. 

15 in such a case, it is preferable to manually set the ref- 
erence parameters. 

[0203] Although in the above example three-dimen- 
sional shape data of the target is created, the method 
can also be applied where it is desired to create pseudo- 
20 three-dimensional data, such as QuickTime VR data, of 
the target. By using the method of the present invention, 
the problem of colors changing when the target is rotat- 
ed and displayed can be avoided. 
[0204] When a real three-dimensional model is creat- 
es ed by inputting a person's face or body and forming a 
model for providing to a customer as described above, 
a certain amount of time is required between the input- 
ting process and completion of the model formation. 
When such a three-dimensional modeling apparatus is 
30 to be placed in, for example, an arcade, it may be pref- 
erable to employ the following embodiment. The de- 
scription will be for a case where a human face is input. 
[0205] The three-dimensional modeling apparatus 
comprises an inputting section for inputting data and a 
35 forming section. These sections can be integrated to- 
gether, or, alternatively can be connected by a network 
as shown in Fig. 26. The inputting section includes a 
memory card issuing section, and the three-dimensional 
modeling apparatus issues a memory card when a cus- 
40 tomer inputs his face data. The memory card stores in- 
formation such as the identification number of the three- 
dimensional shape data of the input face and the actual 
three-dimensional shape data. 

[0206] While the customer waits for the completion of 
45 the model, they are enticed to use other game ma- 
chines. It is preferable to allow the customer to use the 
issued memory card by letting the other machine read 
the contents of the memory card and using the recorded 
three-dimensional shape data for display or for replac- 
50 ing a character within the game. While conventional 
game machines only allow for input of the name when 
a new point record is made, with the present apparatus, 
it is possible and preferable to display the three-dimen- 
sional shape data stored in the memory card along with 
55 the point record. In this manner, a customer can enjoy 
their waiting time and the likelihood of customer dissat- 
isfaction is reduced. 

[0207] It is also preferable to allow for additional re- 
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cording on the memory card for the game points at the 
other game machines. By determining size, coloring pa- 
rameters, or number of products of the real three-dimen- 
sional model to be formed based on the recorded points, 
it is possible to encourage the customer to use more 5 
game machines. 

[0208] In this case, the formation process must be 
started by a start command such as the customer insert- 
ing the memory card into the forming device. 
[0209] Moreover, when the game machines within an 10 
arcade are interconnected by a network, it is preferable 
to store only the identification number in the memory 
card and to distribute the three-dimensional shape data 
to each of the game machines via the network. Alterna- 
tively, it is possible to issue an identification number of 
a customer when he inputs the face data, and let the 
customer input the identification number at each game 
machine so that there is no need for a memory card. 
Specifically, the identification number can be an electri- 
cal mail address or a phone number for a portable 
phone, both of which may be input by the customer, or 
the total sequence number of the customer counted 
from the start of the business, 

[021 0] It is also important to notify the customer when 
the actual formation process has been completed. 
When the game machines within an arcade are connect- 
ed via a network, it is preferable to display a message 
notifying the customer of the completion of the model 
formation on a game machine that the customer is using 
at that time. In consideration of a case where the cus- 
tomer is not using a game machine, it is also preferable 
to separately provide a display panel (completion dis- 
play panel 260) within the arcade fordisptaying the iden- 
tification number for models that have been completed. 
[0211] In addition to the above, it is preferable that the 
phone numberfor a portable phone belongingto the per- 
son requesting the model be input along with the face 
data, and to set the apparatus so that it notifies the cus- 
tomer by calling their portable phone when the formation 
is complete. It is possible to notify the completion by 
sending an electrical mail to a portable phone 261 or a 
portable terminal with an electrical mail capability. 
[0212] In addition, it is also preferableto display, when 
inputting the face data, the earliest time when the real 
three-dimensional model formation can be completed, 
to allow the customer to specify a desired time after the 
earliest time, and to start the formation of the model to 
follow the specified schedule. In this manner, the cus- 
tomercan predict when the real three-dimensional mod- 
el will be available and can obtain the model at the de- 
sired time. 

[0213] When game points earned at other game ma- 
chines affect the resulting real three-dimensional model 
as described above, it is preferable to set the apparatus 
so that the formation starting time is pre-specif ied when 
the face data is input and that the points obtained prior, 
to the starting time affect the resulting real three-dimen- 
sional model. 



[0214] For a customer who wishes to obtain three-di- 
mensional shape data but does not require a real three- 
dimensional model, it is also preferable that the three- 
dimensional shape data be sent via electronic mail after 
the face data is input,. Alternatively, it is also preferable 
that the three-dimensional shape data be saved on a 
server and that an electronic mail be sent to provide 
identification information for accessing the server. In the 
latter case, the customer accesses the server by ftp or 
via the web when the data becomes necessary, inputs 
the sent identification number, and downloads the de- 
sired data. 

[0215] It is also preferable that a delivery destination 
address be input along with the face data, and, when 
the real three-dimensional model is formed, to print and 
output the address for directly delivering the real three- 
dimensional model. After the forming process, the cre- 
ated real three-dimensional model can be packed, the 
printed and output delivery destination label can be af- 
fixed on the package, and the package can be trans- 
ferred to a suitable delivery agent, allowing for a very 
easy delivery procedure. When a plurality of forming 
sections are present in a wide area through a network, 
it may be preferable to transmit the three-dimensional 
shape data to a forming device that is logistically con- 
venient or geographically close based on the delivery 
destination address, to form the model at the selected 
forming device, and to deliver the model to the specified 
delivery destination. By employing such a method, the 
time and cost for delivery of the real three-dimensional 
model to the customer can both be reduced. 
[0216] When the real three-dimensional model is 
formed, rather than forming only the face which has 
been input, by forming a plurality of faces or by forming 
a face with other accessories, an added-value can be 
obtained. For this purpose, the three-dimensional mod- 
eling apparatus is connected to a database via a net- 
work. In the database, pre-stored three-dimensional 
shape data for celebrities, CG synthesized three-dimen- 
sional shape data for animation heroes, or three-dimen- 
sional shape data of friends of the customer, and the 
three-dimensional shape data for accessories such as 
glasses and earrings are stored and managed. 
[0217] The three-dimensional modeling apparatus 
further comprises a user interface section. The custom- 
er can view various data registered in the database and 
specify desired data via the user interlace section. The 
specified data is integrated with the three-dimensional 
shape data of the customer himself, and the generation 
of the real three-dimensional model is performed at the 
forming section. With the above procedure, it is possible 
for the customer to obtain a virtual real three-dimension- 
al model such as, for example, a model in which the cus- 
tomer and a celebrity are together. It is also possible to 
generate a virtual real three-dimensional model where 
the model wears ornaments that the customer does not 
actually wear. ^ 

[0218] Many of such three-dimensional modeling ap- 



20 



25 



30 



35 



40 



45 



50 



15 



BNSDOCID: <EP 1 153638A1_I_> 



29 



EP 1 153 638 A1 



30 



paratuses comprise a plurality of forming devices rather 
than only one forming device. The plurality of forming 
devices can either have identical specifications or dif- 
ferent specifications. As different specification, various 
cases can be considered, such as, for example, different 
size of three-dimensional models that can be formed, 
different materials for the forming object, or different 
method of forming, such as, for example one unit em- 
ploying a milling method and one based on 3D lithogra- 
phy type, for example). 

[0219] When a plurality of forming devices are con- 
nected, it is preferable to schedule these apparatuses. 
Specifically a scheduling section comprises a calculator 
such as a computer, and manages the processing 
progress at each of the connected forming devices and 
information on the data which is waiting to be processed. 
Also, each of the forming devices may store information 
on the average completion time for formation for each 
set of three-dimensional shape data. 
[0220] The three-dimensional shape data : after the 
modeling process at the modeling section , is sent to the 
scheduling section. Based on the operational conditions 
of each of the forming devices, the data is transmitted 
to a suitable forming device. Here, a suitable device 
would normally be the forming device with the shortest 
wait time for the process. It is possible for a customer to 
specify parameters for formation such as the formation 
size or the like at the user interface section. I n this case, 
the scheduling section may, instead of strictly selecting 
the device with the shortest wait time, may select a de- 
vice with the shortest wait time from among devices that 
satisfy the user specified parameters. 
[0221] It may not in all instances be desired that the 
data input and the formation start be close. For example, 
in an application for preparing a statue of deceased for 
funeral use, it is preferable that the data be input while 
the customer is still alive and then be stored in a server. 
A formation request will be made when the statue is 
needed, and the formation will then be started at that 
point. 

[0222] Thus, in this case, a plurality of data, in actu- 
ality, an enormous amount of data, must be stored in the 
server. It is possible to send the formation request elec- 
tronically, such as via an electronic mail and the forming 
section also accepts the delivery destination for deliver- 
ing the product, in addition to the formation request. 
[0223] An apparatus for supplying a real three-dimen- 
sional model to a customer by automatically performing 
processes from three-dimensional input till the forma- 
tion has been described. Next, a method for supplying 
a real three-dimensional mode! to a customer after 
coloring the formed real three-dimensional model will be 
described. 

[0224] The coloring process may be performed auto- 
matically as described before. Here, however, an exam- 
ple will be described wherein the coloring process is 
manually performed in order to allow for more flexible 
coloring to satisfy the customer's wishes. An example 



of this case includes a memorial doll production forfam- 
ily events such as a wedding or a funeral. Specifically, 
the case includes production of "May Dolls" resembling 
a child, production of dolls of a bride and groom for a 
5 wedding, and production of a doll of the deceased to be 
displayed at a funeral. 

[0225] The uncolored real three-dimensional model, 
created by the real three-dimensional modeling appara- 
tus as described above, is sent for coloring process, 
10 which may be performed manually by a coloring crafts- 
man. The coloring craftsmen color the real three-dimen- 
sional model, and the colored real three-dimensional 
model will be delivered to the customer . 
[0226] in order to efficiently complete such coloring 
15 procedure by coloring craftsmen, it is preferable to per- 
form the coloring operation while viewing the three-di- 
mensional data from an specified angle. For this pur- 
pose, in the coloring step, the three-dimensional shape 
data is sent along with the real three-dimensional model 
to be colored. This further requires a display apparatus 
for displaying the object from an arbitrary view point 
based on the sent three-dimensional shape data. 
[0227] The display apparatus corresponds to, for ex- 
ample, a display of a personal computer. The personal 
computer temporarily stores the received three-dimen- 
sional shape data, generates an image from an arbitrary 
point of view based on this data and according to a di- 
rection given by the coloring craftsman, and displays an 
image on the display. Preferably, the display is one ca- 
pable of a three-dimensional display including stereo- 
scopic display. Specifically, examples of the display ap- 
paratus includes a three-dimensional display using a liq- 
uid crystal shutter or changing glasses and a three-di- 
mensional display of no-glass type with a lenticular lens 
or a parallax barrier. 

[0228] It is desirable that the real three-dimensional 
model is directly delivered to the customer after the 
coloring procedure is completed. For this purpose, it is 
possible for a customer to input a desired delivery des- 
tination via the user interface included in the inputting 
section. The delivery destination is printed on a label 
and sent to the coloring craftsman along with the real 
three-dimensional model to be colored. The data can 
also be sent to the coloring craftsman via a network and 
can be printed on the label by the coloring craftsman. 
[0229] It is also preferable that a plurality of coloring 
craftsmen exist with their schedules managed as de- 
scribed before. 

[0230] When such a real three-dimensional model is 
generated, it is also possible to use a component formed 
based on the individual input data only for the head por- 
tion, and to combine the created head component with 
a pre-formed body component. In general, a character- 
istic of a person appears mostly in the head (face) por- 
tion. Therefore, by using a common component for the 
less-distinctive body section, production costs can be 
reduced. For example, the pre-provided component can 
be a body section of a commercially available doll. In 
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such a case, it is preferable to provide a c omponent, 
referred to as a neck component, for incorporating the 
head component with the following characte= istics for in- 
corporating an individually prepared head component to 
a body component. 

1 . The neck component has a shape such that it can 
be affixed to the doll body component. 

2. The neck component has a shape that allows at- 
tachment to the doll head component. 

3. The neck component is rotatabie after attach- 
ment to the doll head component. 

4. The neck component is made of the same mate- 
rial as the doll head component. 

[0231] In other words, as shown in Fig. 27, a doll is 
created by connecting the doll head component, neck 
component, and body component. As with a common 
doll, the head portion is rotatabie with respect to the 
neck. Moreover, although a doll is usually outfitted with 
clothing, at least a portion of the neck is exposed, as is 
the head portion. Therefore, it will be conspicuous if the 
materials for the head and the neck are different, and 
thus, it is desirable that the materials be identical. 
[0232] In addition to the above it is also preferable to 
provide the three-dimensional shape data itself without 
creating a real object model based on the input three- 
dimensional shape data. Specific examples of this ap- 
plication include a dating agency and three-dimensional 
goods catalog for the Internet. 

[0233] When providing data in a field such as a dating 
agency, it is preferable that a restriction be applied for 
data access and for display point of view. In other words, 
only the registered members are allowed to access the 
data and only a display from a display view point that is 
permitted by the person who provided the three-dimen- 
sional shape data is allowed. The restriction is particu- 
larly important for the lauer case, because in general, 
there is a view point that a person do not wish others to 
see from (such as, for example, view point at the bottom 
viewing toward top). 

[0234] It is also prefe rable to distinguish the guest us- 
ers and the registered members on the range of allowed 
view points that can be chosen. In this manner, incentive 
to join the members can be increased. 
[0235] It is also preferable for a plurality of data for 
various facial expressions to be input when inputting the 
data for one person, and to sequentially display three- 
dimensional shape data with a plurality of facial expres- 
sions according to a request from a viewer. Particularly, 
by using common data for the body portion of a person 
and displaying the plurality of face expressions by only 
replacing the head portion, three-dimensional display of 
a plurality of facial expression data for the person can 
be preformed quickly. 

[0236] In addition to the above, it is also preferable 
that voice data be along with the three-dimensional 
shape data. By combining the voice with the three-di- 



mensional shape data, the data can be displayed with 
the voice. With current computer graphics technology, 
it is possible to alter the three-dimensional shape data 
of a face to move the mouth to correspond to the voice 

5 to create an appearance that the three-dimensional dis- 
play is speaking, and such a display is also preferable. 
[0237] Compared to the total volume of all three-di- 
mensional shape data used by, for example, a dating 
agency, in the application for a three-dimensional prod- 

10 uct catalog on the Internet, an enormous amount of data 
must be managed. Therefore, there is a need to select 
a certain items from the enormous amount of data for 
display to a customer. For this purpose, the it is prefer- 
able that the three-dimensional modeling apparatus ac- 

15 cess a database system. 

[0238] As shown in Fig. 28, property information is 
stored in the database along with the input three-dimen- 
sional shape data. The property information varies ac- 
cording to the application. For a dating agency, the prop- 

20 erty information correspond to information such as 
taste, hobby, background, and body size including 
height and weight. In a case for three-dimensional prod- 
uct catalog on the Internet, the property information cor- 
respond to the size of the items, color information, spee- 
ds ification, and price. The size information can also be 
easily and automatically calculated from the obtained 
three-dimensional shape data, andthus, it is possible to 
register the automatically extracted information as the 
property information. 

30 [0239] It is preferable that, in addition to the three-di- 
mensional shape data itself, a thumbnail image of the 
data also be managed by the database. This is prefer- 
able because there is a demand for a customer to view 
the general image of an item by a thumbnail image after 

35 the customer has narrowed down from the items regis- 
tered in the database based on specific keywords. Ex- 
amples of methods for generating a thumbnail image in- 
clude: 

40 1 . generating an image from a specific point of view 
according to the three-dimensional shape data and 
using the image as the thumbnail image; 

2. generating a plurality of images from a plurality 
of points of view according to the three-dimensional 

45 shape data, integrating the plurality of images into 
a series of animation, and obtaining an animated 
thumbnail image; 

3. reducing a two-dimensional image used during 
the three-dimensional shape data input and using 

so the image as the thumbnail image; and 

4. generating a reduced thee-dimensional shape 
data by further reducing data from the original three- 
dimensional shape data and using the image as the 
thumbnail image. 

55 

[0240] When undetermined access to the data via a 
network is made possible, it may be preferable to em- 
ploy a system where the number of accesses to each 
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data is recorded and charges are determined based on 
the recorded data. The database has information on the 
charge/refund subject related to each data. The number 
of accesses for the data are collected every month and 
charge/refund processes are executed for the charge/ s 
refund subject according to the number of accesses. In 
a case of a product catalog , a method can be considered 
where a greater charge is associated with items with 
more accesses, because the number of accesses indi- 
cate that significant advertising effect was presented. 10 
On the other hand, there is another system where a pre- 
determined amount of money is paid to the supplier of 
the data system when registering to the database, and 
wherein this amount increases the amount of refund 
with an increase in the number of accesses. The latter is 
case is based on a concept that data which utilizes the 
storing space, but is not accessed should be penalized. 
In some cases, the refund may be considered as a boun- 

[0241] In the above, various examples are described. 20 
In any of the cases for supplying the three-dimensional 
shape data, usage of a three-dimensional display appa- 
ratus as the display side is preferable. A three-dimen- 
sional display apparatus is one of the most suited appa- 
ratus for displaying three-dimensional shape data. It is 25 
effective to input voice with the th ree-dimensiona! shape 
data and to input three-dimensional movement informa- 
tion along with the shape information. Because human 
body movements are produced at the joints, it is possi- 
ble to divide the input three-dimensional shape data into 30 
parts at the joint sections. By recording the relative 
movement information for each of the parts, movement 
information can effectively be recorded. When prepar- 
ing a three-dimensional image for the customer, the 
three-dimensional image can be shown efficiently by 35 
displaying the motion of the three-dimensional shape 
data so that the display moves part-by-part based on 
the input movement data. 

[0242] As described above, according to the present 
invention, a practical system for effectively and efficient- 40 
ly providing automatically generated real three-dimen- 
sional model and three-dimensional shape data to a 
customer can be realized. 



45 

Claims 

1. A three-dimensional modeling apparatus for sup- 
plying a three-dimensional model of a target to a 
customer, comprising: 50 

an input section for inputting three-dimensional 
shape data of said target; 
a formation section for forming a real three-di- 
mensional model based on the obtained three- 55 
dimensional shape data; and 
a recording medium issuing section for issuing 
a recording medium in which information can 



6. 



7. 



9. 



be written to or read from, when said three-di- 
mensional shape data is input; wherein 
said recording medium can be read by said 
three-dimensional modeling apparatus and/or 
a third-party apparatus other than said three- 
dimensional modeling apparatus . 

A thi ;e-dimensional modeling apparatus of claim 1 , 
w heroin said recording medium is capable of re- 
cording data identification information. 

A tr ' ee dimensional modeling apparatus of claim 1 , 
who-ei: said recording medium allows new infor- 
mation be written by said three-dimensional mod- 
eling- .p- *ratus and/or an third-party apparatus oth- 
er nar said three-dimensional modeling 
appar-uu.. . 

A three- dimensional modeling apparatus of claim 3, 
whereii sb. d third-party apparatus is an entertain- 
ment de ict: and said new information is a score of 
the ente tai -ment provided by said entertainment 
device, s..i id score being evaluated by the enter- 
tainment cev ^e. 

A three-din-en ^onal modeling apparatus of claim 4 , 
further com urit a recording medium information 
re-input sector, or re-reading said new information 
from said recor;::?ng medium, said new information 
having been «vrii\en by said third-party apparatus. 

Athree-dimenstor-al modeling apparatus of claim 5, 
wherein the fcrrr ntion parameters are changed 
based on the in r orr*ation input from said recording 
medium informfciio. re-input section and a real ob- 
ject model is formed from the changed formation 
parameters. 

A three-dimensional r odeling apparatus of claim 1 , 
wherein three-dimens onal shape data which is in- 
put by said input se tic - is recorded on said record- 
ing medium. 

A three-dimensional noeling apparatus of claim 7, 
wherein said third-peiy spparatus is an entertain- 
ment device which leacs the three-dimensional 
shape data recorded o:> s^d recording medium and 
executes an entertainr en program of said enter- 
tainment device. 

A three-dimensional modeling apparatus for sup- 
plying a three-dimensional model of a target to a 
customer, comprising: 

an input section for inp tti- q three-dimensional 
shape data of said tare rt; 
a formation sectiqrj for >rp «ing a real three-di- 
mensional model based ne obtained three- 
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dimensional shape data; 
a communication section which can communi- 
cate with said three-dimensional modeling ap- 
paratus and/or a third-party apparatus other 
than said three-dimensional modeling appara- 
tus via a network; and 

a formation process progress information 
transmitting section for transmitting the 
progress information of the formation process 
at said formation section to said apparatus. 

10. A three-dimensional modeling apparatus of claim 9, 
wherein said formation process progress informa- 
tion of said formation section includes formation 
completion time information and said time informa- 
tion is displayed at a display section provided on 
said apparatus. 

11. A three-dimensional modeling apparatus of claim 9, 
wherein said communication section receives the 
information at said apparatus and changes param- 
eters for forming the real three-dimensional model 
based on said information, and said formation sec- 
tion forms the real three-dimensional model based 
on the changed parameters. 

1 2. A three-dimensional modeling apparatus of claim 9, 
wherein said three-dimensional shape data is trans- 
mitted to said apparatus via said communication 
section. 

13. A three-dimensional modeling apparatus of claim 
12, wherein said apparatus is an entertainment de- 
vice and said transmitted three-dimensional model 
shape data is used by said entertainment device. 

14. A three-dimensional modeling apparatus for sup- 
plying a three-dimensional model of a target to a 
customer, comprising: 

an input section for inputting three-dimensional 
shape data of said target; 
a formation section for forming a real three-di- 
mensional model based on the obtained three- 
dimensional shape data; 
a contacting means input section for inputting 
information contacting means for said custom- 
er; and 

an information notifying section for notifying 
modeling information to said information con- 
tacting means. 

15. A three-dimensional modeling apparatus of claim 
14, wherein said information contacting means for 
the customer is a cellular phone, the phone number 
for said cellular phone is input at said contacting 
means input section, and said modeling information 
is the progress information of formation process at 
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said formation section. 

16. A three-dimensional modeling apparatus of claim 
14, wherein said information contacting means for 
5 the customer is an electronic mail, the mail address 
for the electronic mail is input at said contacting 
means input section, and said modeling information 
is formation completion information. 

10 17. A three-dimensional modeling apparatus of claim 
16, wherein said modeling information is identifica- 
tion information for accessing a server which stores 
the three-dimensional shape data, in addition to be- 
ing said formation completion information. 

15 

18. A three-dimensional modeling apparatus of claim 
16, wherein said modeling information is three-di- 
mensional shape data. 

20 19. A three-dimensional modeling apparatus of either 
claim 9 or claim 14, further comprising: 

an estimated completion time display section 
for displaying the shortest estimated time for 

25 the formation completion; and 

a formation completion time specifying section 
for specifying a time after the shortest formation 
completion time, wherein 
a schedule for the formation process is deter- 

30 mined to accommodate the completion time in- 

dicated at said formation completion time spec- 
ifying section by said customer. 

20. A three-dimensional modeling apparatus of claim 
35 1 9, further comprising a process start receiving sec- 
tion for receiving start time for the formation proc- 
ess, wherein the formation process is started ac- 
cording to the start time indicated by said customer. 

40 21 . A three-dimensional modeling apparatus of either 
claim 9 or 14, further comprising: 

an address means input section for inputting 
delivery destination address for said real three- 
45 dimensional model; and 

a delivery destination printing section for print- 
ing said delivery destination address. 

22. A three-dimensional modeling apparatus of either 
50 claim 9 or 1 4, further comprising a formation section 

selecting section, connected to a plurality of forma- 
tion sections via a network, for selecting a suitable 
formation section based on said input address, 
wherein the three-dimensional shape data is trans- 
55 mitted to said selected formation section. 

23. Athree-dimensional modeling apparatus of any one 
of claims 1 , 9, and 14, further comprising a commu- 
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nication section which can communicate with a da- 
tabase section for storing the three-dimensional 
shape data via a network, wherein said three-di- 
mensional shape data is recorded at said database 
section. 

24. A three-dimensional modeling apparatus of claim 
23, further comprising a three-dimensional shape 
data synthesizing section for reading three-dimen- 
sional shape data p re-stored in said database sec- 
tion and generating new three-dimensional shape 
data obtained by integrating a plurality of three-di- 
mensional shape data. 

25. A three-dimensional modeling apparatus of an one 
of Claims 1 , 9, and 14, further comprising: 

a plurality of formation sections, and 
a server section for managing the three-dimen- 
sional shape data input from said input section, 
wherein 

said formation section is selected at said server 
section for assigning said three-dimensional 
shape data input from said input section. 

26. A three-dimensional modeling apparatus of claim 
25, wherein the selection of said formation section 
is performed based on the operation conditions at 
said plurality of formation sections. 

27. A three-dimensional modeling apparatus of claim 
25, further comprising a parameter specifying sec- 
tion for the customer to specify the material or size 
of the object to be formed, wherein the selection of 
said formation section is performed based on the 
parameters specified at said parameter specifying 
section. 

28. A three-dimensional modeling apparatus of claim 
25, wherein at least two of said input section, for- 
mation section, and server section are connected 
via a network. 

29. A three-dimensional modeling apparatus of claim 
25, further comprising a display section for display- 
ing an image of said three-dimensional shape data 
seen from an arbitrary point of view. 

30. A three-dimensional modeling apparatus of claim 
25, further comprising a coloring section for coloring 
the real three-dimensional model based on said 
three-dimensional shape data. 

31. A three-dimensional modeling apparatus of claim 
29, wherein said display section is a display device 
capable of three-dimensional display including ster- 
eo-scopic display. 
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32. A three-dimensional modeling apparatus for sup- 
plying a three-dimensional model of a target to a 
customer, comprising: 

5 a data input section for inputting three-dimen- 

sional shape data of said target; and 
a formation section for forming a real three-di- 
mensional model based on the obtained three- 
dimensional shape data; wherein 

10 said real three-dimensional model comprises a 

plurality of parts, and a portion of the parts is 
selected from among a plurality of pre-provided 
components based on the measured three-di- 
mensional shape data of said target. 

15 

33. A three-dimensional modeling apparatus of claim 

32, wherein said selected component is further 
processed. 

20 34. a three-dimensional modeling apparatus of claim 

33, wherein 

said real three-dimensional model is construct- 
ed from at least three parts, 
25 the first and second parts are constructed from 

an identical first material, 
the third part is constructed from a material dif- 
ferent from said first material, 
three-dimensional shape data which corre- 
30 spond to at least said first part is input at said 

input section, 

the first part is formed at said formation section 
based on said input three-dimensional shape 
data, and 

35 said formed first and second parts and second 

and third parts are respectively integrated by 
joining respective parts together. 

35. A three-dimensional modeling apparatus for sup- 
40 plying a three-dimensional model of a target to a 

customer, comprising: 

an input section for inputting three-dimensional 
shape data of said target; 
45 a recording section for recording a plurality of 

three-dimensional shape data which are input 
from said input section; 

an identification number input section for input- 
ting an identification number which is used for 
50 displaying said recorded three-dimensional 

shape data; 

an identification number judging section for 
judging said input identification number; and 
a data outputting section for outputting said re- 
55 corded three-dimensional shape data. 

36. A three-dimensional modeling apparatus of claim 
35, wherein the data can be displayed from view- 
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points within a predetermined range for the display 
process of the three-dimensional shape data at said 
data outputting section. 

'37. A three-dimensional modeling apparatus of claim 5 
36, wherein 

there exist a plurality of said predetermined 
points of view, 

one of the plurality of points of view is selected 10 
based on said identification number, and 
the three-dimensional shape data is displayed 
at said data outputting section based on the se- 
lected point of view. 

75 

38. A three-dimensional modeling apparatus of claim 
35, further comprising a modifying section for read- 
ing and modifying said three-dimensional shape da- 
ta recorded at said recording section, and for re- 
cording the modified data to said recording section. 20 

39. A three-dimensional modeling apparatus of claim 
35, wherein at least two of said input section, re- 
cording section, identification number input section, 
identification number judging section, data output- 25 
ting section, and modifying section are connected 

via a network. 

40. A three-dimensional modeling apparatus of claim 

35, wherein said data outputting section is a display 30 
device capable of three-dimensional display includ- 
ing stereo-scopic display. 

41. A three-dimensional modeling apparatus of claim 

35, wherein 35 

said target is a person, 

voice of said target person is also input at said 
input section, 

said voice is recorded at said recording section *o 
in addition to said three-dimensional shape da- 
ta, and 

said data outputting section is capable of out- 
putting said three-dimensional shape data and/ 
or said voice. 45 

42. A three-dimensional modeling apparatus of claim 
35, wherein 

a plurality of facial expressions are input at said so 
input section for one person, 
said recording section records said plurality of 
facial expressions, and 

said displaying section is capable of sequen- 
tially displaying said plurality of face expres- 55 
sions in response to directions by said custom- 
er. 



43. A three-dimensional modeling apparatus for sup- 
plying a three-dimensional model of a target to a 
customer, comprising: 

an input section for inputting three-dimensional 
shape data of said target; 
a database section for registering the three-di- 
. mensional shape data input from said input 
section and its property information; and 
a display section for displaying said registered 
three-dimensional shape data. 

44. A three-dimensional modeling apparatus of claim 

43, further comprising a modifying section for read- 
ing and modifying said three-dimensional shape da- 
ta registered at said database section, and for re- 
registering the modified data to said database sec- 
tion. 

45. A three-dimensional modeling apparatus of claim 

44, wherein a thumbnail image of each of three-di- 
mensional shape data is registered at said data- . 
base section. 

46. A three-dimensional modeling apparatus of claim 

45, wherein said thumbnail image is produced by 
assigning a point of view for said three-dimensional 
shape data and obtaining a two-dimensional image 
seen from the assigned point of view. 

47. A three-dimensional modeling apparatus of claim 
45, wherein said thumbnail image is produced by 
assigning a plurality of points of view for said three- 
dimensional shape data and obtaining a series of 
two-dimensional images seen from these points of 
view. 

48. A three-dimensional modeling apparatus of claim 
45, wherein said thumbnail image is produced by 
selecting one of the two-dimensional images that 
were taken when inputting said three-dimensional 
shape data, and reducing the selected two-dimen- 
sional image. 

49. A three-dimensional modeling apparatus of claim 
45, wherein said thumbnail image is a reduced 
three-dimensional shape data obtained by reducing 
the data in said three-dimensional shape data. 

50. A three-dimensional modeling apparatus of claim 
43, wherein number of accesses are also registered 
for and in addition to each of the data registered at 
said database section. 

51. A three-dimensional modeling apparatus of claim 
50, further comprising a charging device, wherein 
charge process for charge targets registered in 
each database is executed based on said number 
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of accesses. 

52. A three-dimensional modeling apparatus of claim 
51 , further comprising a refund device, wherein re- 
fund process for refund targets registered in each 5 
database is executed based on said number of ac- 
cesses. 

53. A three-dimensional modeling apparatus of claim 

43, wherein said data display section is a display 10 
device capable of three-dimensional display includ- 
ing stereo-scopic display. 

54. A three-dimensional modeling apparatus of any one 

of claims 1 , 9, 14, 32, 35, and 43, further comprising: is 

a motion input section for inputting motion in- 
formation of said three-dimensional shape data 
input from said input section; 

a dividing section for dividing said three-dimen- 20 
sional shape data input from said input section 
into a plurality of portions; and 
a motion recording section for linking and re- 
cording said motion information and said plu- 
rality of divided portions. 25 
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ABSTRACT : PROBLEM TO BE SOLVED: To provide a compound eye camera by which a moving 

image is displayed while a display rate is set equal to a photographing rate in the case that 
a couple of images picked up by the compound eye camera are displayed on one image 
display section simultaneously. 

SOLUTION: In the case that a switching section 31 is thrown to the position shown by 
dotted lines, vertical interleaving is conducted by a processing circuit 26 of the compound 
camera and an image signal is compressed 1/2 in the vertical direction and the image is 
. compressed 1/2 in the horizontal direction by a horizontal direction compression section 
35, then left/right image signals are written in a video RAM (VRAM) 28 at the same rate as 
the photographing rate. The left and right image signals 34, 304 picked up by two CCDs 
as above are stored in the VRAM 28 as a panoramic image displayed on a liquid crystal 
display device. The panoramic image written in the VRAM 28 is displayed on the liquid 
crystal display device via a liquid crystal display controller. 

COPYRIGHT: (C)1998,JPO 



BNSDOCID: <JP 410336705A_AJ_> 



THIS PAGE BLANK <uspto) 



EUROPEAN PATENT OFFICE 



Patept Abstracts of Japan 



PUBLICATION NUMBER 
PUBLICATION DATE 

APPLICATION DATE 
APPLICATION NUMBER 

APPLICANT : SONY CORP; 

INVENTOR : KAMAYA NAOKI; 



2001251403 
14-09-01 

07-03-00 
2000062257 



INT.CL. 



TITLE 



H04M 1/02 H04Q 7/38 H04M 1/00 
H04N 7/14 H04N 13/04 

TELEPHONE DEVICE 




ABSTRACT : PROBLEM TO BE SOLVED: To display fine information such as character information by 
an information display part and to display information of a human, a body, etc., as a 
stereoscopic image by a stereoscopic image display part by constituting the stereoscopic 
image display part which can make a stereoscopic display as only part of the information 
display part. 

SOLUTION: The telephone device equipped with a liquid crystal display which can display 
information is provided the stereoscopic image display part which can information as a 
stereoscopic image as part of the liquid crystal display 20. 
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